Transfer nonnegative matrix factorization for image representation

Nonnegative Matrix Factorization (NMF) has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain. However, when labeled and unlabeled images are sampled from different distributions, they may be quantized into different basis vector space and represented in different coding vector space, which may lead to low representation fidelity. In this paper, we investigate how to extend NMF to cross-domain scenario. We accomplish this goal through TNMF - a novel semi-supervised transfer learning approach. Specifically, we aim to minimize the distribution divergence between labeled and unlabeled images, and incorporate this criterion into the objective function of NMF to construct new robust representations. Experiments show that TNMF outperforms state-of-the-art methods on real dataset

GC-Flow: A Graph-Based Flow Network for Effective Clustering

Graph convolutional networks (GCNs) are \emph{discriminative models} that directly model the class posterior $p(y|\mathbf{x})$ for semi-supervised classification of graph data. While being effective, as a representation learning approach, the node representations extracted from a GCN often miss useful information for effective clustering, because the objectives are different. In this work, we design normalizing flows that replace GCN layers, leading to a \emph{generative model} that models both the class conditional likelihood $p(\mathbf{x}|y)$ and the class prior $p(y)$. The resulting neural network, GC-Flow, retains the graph convolution operations while being equipped with a Gaussian mixture representation space. It enjoys two benefits: it not only maintains the predictive power of GCN, but also produces well-separated clusters, due to the structuring of the representation space. We demonstrate these benefits on a variety of benchmark data sets. Moreover, we show that additional parameterization, such as that on the adjacency matrix used for graph convolutions, yields additional improvement in clustering.Comment: ICML 2023. Code is available at https://github.com/xztcwang/GCFlo

DyExplainer: Explainable Dynamic Graph Neural Networks

Graph Neural Networks (GNNs) resurge as a trending research subject owing to their impressive ability to capture representations from graph-structured data. However, the black-box nature of GNNs presents a significant challenge in terms of comprehending and trusting these models, thereby limiting their practical applications in mission-critical scenarios. Although there has been substantial progress in the field of explaining GNNs in recent years, the majority of these studies are centered on static graphs, leaving the explanation of dynamic GNNs largely unexplored. Dynamic GNNs, with their ever-evolving graph structures, pose a unique challenge and require additional efforts to effectively capture temporal dependencies and structural relationships. To address this challenge, we present DyExplainer, a novel approach to explaining dynamic GNNs on the fly. DyExplainer trains a dynamic GNN backbone to extract representations of the graph at each snapshot, while simultaneously exploring structural relationships and temporal dependencies through a sparse attention technique. To preserve the desired properties of the explanation, such as structural consistency and temporal continuity, we augment our approach with contrastive learning techniques to provide priori-guided regularization. To model longer-term temporal dependencies, we develop a buffer-based live-updating scheme for training. The results of our extensive experiments on various datasets demonstrate the superiority of DyExplainer, not only providing faithful explainability of the model predictions but also significantly improving the model prediction accuracy, as evidenced in the link prediction task.Comment: 9 page

Efficient ab initio many-body calculations based on sparse modeling of Matsubara Green's function

This lecture note reviews recently proposed sparse-modeling approaches for efficient ab initio many-body calculations based on the data compression of Green's functions. The sparse-modeling techniques are based on a compact orthogonal basis representation, intermediate representation (IR) basis functions, for imaginary-time and Matsubara Green's functions. A sparse sampling method based on the IR basis enables solving diagrammatic equations efficiently. We describe the basic properties of the IR basis, the sparse sampling method and its applications to ab initio calculations based on the GW approximation and the Migdal-Eliashberg theory. We also describe a numerical library for the IR basis and the sparse sampling method, irbasis, and provide its sample codes. This lecture note follows the Japanese review article [H. Shinaoka et al., Solid State Physics 56(6), 301 (2021)].Comment: 26 pages, 10 figure

First-principles study of oxygen vacancy defects in orthorhombic Hf0.5_{0.5}Zr0.5_{0.5}O2_2/SiO2_2/Si gate stack

The gate defect of the ferroelectric HfO$_2$-based Si field-effect transistor (Si FeFET) plays a dominant role in its reliability issue. The first-principles calculations are an effective method for the atomic-scale understanding of gate defects. However, the first-principles study on the defects of FeFET gate stacks, i.e., metal/orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si structure, has not been reported so far. The key challenge is the construction of metal/orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si gate stack models. Here, we use the Hf$_{0.5}$Zr$_{0.5}$O$_2$(130) high-index crystal face as the orthorhombic ferroelectric layer and construct a robust atomic structure of the orthorhombic-Hf$_{0.5}$Zr$_{0.5}$O$_2$/SiO$_2$/Si gate stack without any gap states. Its high structural stability is ascribed to the insulated interface. The calculated band offsets show that this gate structure is of the type-I band alignment. Furthermore, the formation energies and charge transition levels (CTLs) of defects reveal that the oxygen vacancy defects are more favorable to form compared with other defects such as oxygen interstitial and Hf/Zr vacancy, and their CTLs are mainly localized near the Si conduction band minimum and valence band maximum, in agreement with the reported experimental results. The oxygen vacancy defects are responsible for charge trapping/de-trapping behavior in Si FeFET. This work provides an insight into gate defects and paves the way to carry out the first-principles study of ferroelectric HfO$_2$-based Si FeFET.Comment: 18 pages, 5 figure

Deep air learning: Interpolation, prediction, and feature analysis of fine-grained air quality

The interpolation, prediction, and feature analysis of fine-gained air quality are three important topics in the area of urban air computing. The solutions to these topics can provide extremely useful information to support air pollution control, and consequently generate great societal and technical impacts. Most of the existing work solves the three problems separately by different models. In this paper, we propose a general and effective approach to solve the three problems in one model called the Deep Air Learning (DAL). The main idea of DAL lies in embedding feature selection and semi-supervised learning in different layers of the deep learning network. The proposed approach utilizes the information pertaining to the unlabeled spatio-temporal data to improve the performance of the interpolation and the prediction, and performs feature selection and association analysis to reveal the main relevant features to the variation of the air quality. We evaluate our approach with extensive experiments based on real data sources obtained in Beijing, China. Experiments show that DAL is superior to the peer models from the recent literature when solving the topics of interpolation, prediction, and feature analysis of fine-gained air quality